XR System Intends to Support Sight-Impaired Workers
For people with visual impairments, difficulties with everyday work tasks can arise from environmental barriers rather than a lack of competence. Researchers from the Karlsruhe Institute of Technology (KIT) and the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB) are developing a mobile assistance system intended to help address these barriers.
The AssistiveXR4Work project combines extended reality (XR) glasses with artificial intelligence to adapt the work environment to individual vision requirements. The system is intended to help people with sight impairments work more independently and make it easier for qualified professionals to secure employment.
Daily tasks can present obstacles for skilled workers with visual impairments. Reading a device label, deciphering a handwritten note or finding a way around an unfamiliar space can be time-consuming or impossible without assistance. Conventional aids often reach their limits beyond the computer workstation.
The AssistiveXR4Work system is designed to automatically recognise or magnify visual information, display it with high contrast or read it aloud, depending on individual vision requirements. It can, for example, recognise and magnify whiteboard content during a meeting. It can also convert handwritten notes into text and further process them without human intervention.
In engineering work environments, the system can assist with orientation and highlight relevant components. These functions are intended to provide support in workplaces where workers may need to interpret visual information or navigate unfamiliar surroundings.
The system uses extended-reality glasses equipped with a high-resolution camera. The glasses continuously capture the surroundings and digitally adapt the display to individual vision requirements. They are able to enhance contrast, magnify content, recognise text and read it aloud on demand.
An integrated artificial intelligence system also assists with orientation and recognises objects in the surrounding environment. A human helper can be connected through a livestream when required.
The project also addresses the shortage of skilled workers. Many highly skilled employees lose their jobs or have to shift their field of activity when their vision deteriorates. AssistiveXR4Work is designed to help them continue their careers even as their vision deteriorates.
According to the Institute for Employment Research (IAB), a research institution operated by the Federal Employment Agency, severe disabilities are widespread in Germany, with most occurring during a person’s professional life. For people affected by a severe disability, the probability of still being part of the labour force five years after the disability occurs is reduced by roughly 16 percentage points.
The AssistiveXR4Work project plans to construct an operational demonstrator over the next four years and then test it in practice.
